MUNDO and TRIDENT GeoAI v2 explanation

My name is Yui Matsumura

Please call me yuiseki.

My current job

Product manager, Gyazo, Helpfeel Inc. (2022-09-present)

I am not involved in GIS in my work, but I am involved in several GIS projects through OSS activities

What is MUNDO?

MUNDO

stands for Model United Nations Development Operation

The objective is

To maximize the potential of humanity

The method is

To design a majorly upgraded version of the United Nations

To achieve this

MUNDO actively adopts

Blockchain

As for blockchain, Mr. Fujimura is up to the challenge.

Robotics and Drone

As for drones, Mr. Furuhashi is up to the challenge.

Artificial Intelligence

Virtual Reality and Augmented Reality

Internet of Things

Quantum Computing

Brain Machine Interface

Microsatellite

Bio Technology

Fusion Reactor

...and any other potentially available technologies

You are probably thinking...

Are you serious?

Are you kidding me?

My answer

I'm serious.

I'm not kidding.

To prove it, I will show you a demonstration.

Demonstration of TRIDENT

What is TRIDENT?

TRIDENT is, an unofficial UN dedicated interactive information retrieval and humanity assistance system

yuiseki.icon

code:prompt

What is the United Nations in a nutshell?

TRIDENT.icon

yuiseki.icon

code:prompt

Who is the current Secretary General of the United Nations?

TRIDENT.icon

yuiseki.icon

code:prompt

Where is the headquarters of the UNHCR?

TRIDENT.icon

Demonstration of TRIDENT GeoAI v2

What is TRIDENT GeoAI?

TRIDENT GeoAI is, an interactive geospatial situation awareness empowerment system

AI that can generate maps based on conversation with the user

yuiseki.icon

code:prompt

Show UN facilities in New York City.

TRIDENT.icon

yuiseki.icon

code:prompt

Add hotels, police stations and train stations to this map.

TRIDENT.icon

yuiseki.icon

code:prompt

Show military installations in South Sudan.

TRIDENT.icon

yuiseki.icon

code:prompt

Add hospitals and shelters to this map.

TRIDENT.icon

yuiseki.icon

code:prompt

Show only military installations, after all.

TRIDENT.icon

yuiseki.icon

code:prompt

Limit the display area to the capital.

TRIDENT.icon

yuiseki.icon

code:prompt

Show military installations in Sudan and South Sudan.

TRIDENT.icon

yuiseki.icon

code:prompt

Show military installations, hospitals, schools, and shelters in Sudan's capital.

TRIDENT.icon

yuiseki.icon

code:prompt

After all, show only military installations.

TRIDENT.icon

NOTE:

I am releasing TRIDENT and TRIDENT GeoAI as open source software under a fully public domain license

NOTE:

I only present examples that work.

There are a lot of things that can go wrong.

My current job

Product manager, Gyazo, Helpfeel Inc. (2022-09-present)

My contributions to humanity via OSS

motivation

In the event of a disaster, you still need to be able to print out paper maps and hand them out!

motivation

I want to enhance OpenStreetMap's building address information in an easy and fun way!

motivation

I want to host a vector tile map for free!

motivation

I would like to do a more flexible and easy style definition for vector tile maps!

motivation

We want to respond to disasters by using digital maps even if the communication environment is unstable at the time of a disaster!

motivation

We would like to have a digital map of the latest status of UN peacekeeping operations and UN development!

Technical prerequisite knowledge

text-davinci-003

PromptTemplate, Chain

Agent, Tool

ReAct

Required for interactive web map UI development

Has anyone touched the Overpass API at all?

The Overpass API (formerly known as OSM Server Side Scripting, or OSM3S before 2011) is a read-only API that serves up custom selected parts of the OSM map data. It acts as a database over the web: the client sends a query to the API and gets back the data set that corresponds to the query.

Unlike the main API, which is optimized for editing, Overpass API is optimized for data consumers that need a few elements within a glimpse or up to roughly 10 million elements in some minutes, both selected by search criteria like e.g. location, type of objects, tag properties, proximity, or combinations of them. It acts as a database backend for various services.

In Tokyo

Search Hospitals

code:overpass

(

);

out geom;

Points

If you know the Area and Tags, you can assemble an Overpass API query.

The events leading up to TRIDENT GeoAI v2

Backgrounds

https://gyazo.com/0b67931b29be63ee9ed63c90a7c78165

https://gyazo.com/18f0232cf9f6308b349a5a44ec97ba5c

Key findings

What are Concerns in MUNDO?

You can call it Theme or Subject of the map, but I call it Concerns out of respect for Rails.

How MUNDO handles Concerns

Simplified Overpass API queries as YAML files

Generate Overpass API queries based on the description in the YAML file

OpenStreetMap has a policy of "not handling temporary information"

For example, incidents, accidents, disasters, disputes, road closed, etc.

MUNDO has the ability to specify the OpenStreetMap place ID in the YAML file and add a place tag

NOTE

Temporary concerns must be maintained manually

Volunteers are needed to help maintain temporary concerns!

https://www.youtube.com/watch?v=XEM5qz__HOU

If this is not a bluff, it's too great to be true....

The AI detects an attack on the base, analyzes enemy forces, analyzes the battlefield terrain and troop deployment to find the shortest route, checks friendly forces, draws up an optimal repulse plan, and says, "Ready to execute. Shall I send the order?"

TRIDENT has a limited GeoAI function that displays a map if a place name is included in the results of a question-and-answer session.

Problems with TRIDENT's [* limited GeoAI functionality

Learn patterns of prompting techniques for successful output to LLMs.

Prompts should be written in English.

It is important to think of multiple prompts that work well together, rather than a single prompt that solves everything.

Implement GeoAI as an Agent in LangChain.js

The Agent was given the following three Tools

Tool to output Area for Overpass API queries

Tool to output Tags for Overpass API queries

Tools to output final Overpass API query

Problems with v1

Slow response

Cannot handle multiple Areas or multiple Concerns

Short lived because I came up with v2 and implemented and released it about 3 days after release.

Improvements in v2

response is fast

Can handle multiple Areas and multiple Concerns

Interactive

Changes from v1

Rethinking data structures to represent complex maps

I stopped using LangChain.js Agent.

TRIDENT GeoAI v2 Data Structure

Backgrounds

The following are required to draw a web map

Recent web maps have separated these, allowing for the creation of flexible looking maps.

Conversely, when drawing, we need to tie them together.

Overview of Data Structure

Concerns can each have a style

Examples

code:geoai

Target Area A: Taito Ward

Interest A: Train station

Appearance: Gray

Interest B: Hospital

Appearance: Pink

Target Area B: Bunkyo-ku

Interest A: Station

Appearance: Gray

Interest B: Hospital

Appearance: Pink

TRIDENT GeoAI v2 three-layer architecture

It has the following three layers

surface layer

The surface layer is dedicated to naturally interacting with the user to find out what kind of map they want to create.

inner layer

The inner layer specializes in analyzing the entire history of user interaction with the surface layer and outputting the information needed to create complex maps as an intermediate language.

deep layer

The deep layer specializes in generating Overpass API queries based on the intermediate language output by the inner layer

Why am I making this complicated?

I want to do dialogue responses asynchronously.

I want to analyze dialogues asynchronously.

I'd also like to make them think about Overpass API queries asynchronously.

I want to make requests to the Overpass API asynchronously.

TRIDENT GeoAI v1 was slow because of,

It was too late to have LLMs output complex and huge Overpass API queries.

Requesting complex and huge Overpass API queries to the Overpass API, with slow responses

Surface Layer

The surface layer is an "interactive online map building assistant"

It's a mediocre dialogue prompt.

Inner Layer

The inner layer is a "dialogue history analysis assistant dedicated to digital map construction"

The inner layer analyzes the history of the user's interaction with the surface layer

If geospatial information cannot be identified from the dialogue history, give up appropriately

The inner layer does not directly output Overpass API queries

The inner layer builds the Overpass API query and outputs the information needed to render the results in an intermediate language

Intermediate language is prefixed + newline delimited instead of JSON

Format of the intermediate language output by the inner layer

code:geoai

ConfirmHelpful: text that meanings "Mapping has been completed. Do you have any other requests? Have we been helpful to you?", MUST be the last language written by the human

EmojiForConcern: emoji best suited to expressing specific concern, MUST be unique for each concern

ColorForConcern: color name best suited to expressing specific concern, MUST be unique for each concern, should be one of the name of Web Safe Color

Area: geospatial area mentioned by human

AreaWithConcern: pair of geospatial area and concern mentioned by human

... (You MUST ALWAYS output only one ConfirmHelpful)

... (this Area/AreaWithConcern/EmojiForConcern/ColorForConcern can repeat N times)

This is a strong reference to ReAct

Part of the ReAct prompt:

code:react

Thought: you should always think about what to do

Action Input: the input to the action

Observation: the result of the action

... (this Thought/Action/Action Input/Observation can repeat N times)

Specific examples of intermediate languages

'Make a map of military installations in Sudan and South Sudan.'

code:geoai

EmojiForConcern: military facility, 🪖

ColorForConcern: Military facility, pink

Area: Sudan

Area: South Sudan

AreaWithConcern: Sudan, military facility

AreaWithConcern: South Sudan, Military facility

"Make a map of stations and hospitals in Taito and Bunkyo wards."

Or, "Make a map of stations and hospitals in Taito Ward, and add Bunkyo Ward to it."

code:geoai

EmojiForConcern: Station, 🚉

EmojiForConcern: hospital, 🏥

ColorForConcern: station, gray

ColorForConcern: hospital, coral

Area: Taito-ku

Area: Bunkyo-ku

AreaWithConcern: Taito-ku, station

AreaWithConcern: Taitung, hospital

AreaWithConcern: Bunkyo-ku, station

AreaWithConcern: Bunkyo-ku, Hospital

Deep Layer

AreaWithConcern: Bunkyo-ku, Hospital

The deep layer is "an OpenStreetMap and Overpass API expert that outputs Overpass API queries precisely based on the intermediate language."

TRIDENT GeoAI v2 Known Issues, Limitations, and Future Prospects

If you specify the exact name of the building, the map drawing will almost certainly fail.

Since the intermediate language only has the concepts of Areas and Concerns, it is not possible to express Points.

It is quite difficult to get a specific building exactly right with the Overpass API.

difficult even for a human being

I want to have PostGIS queries written.

LLMs can also generate SQL

I want to have PostGIS queries written based on user interaction history.

More advanced and precise analysis of geospatial information, such as Palantir AIP, should be possible

Other Topics of Interest

Motivation

Transcribing descriptions of geographic features in natural language based on OpenStreetMap tags

Make it Embedding

Then it should be possible to do a vector search for geographical objects that are semantically close to the natural language sentences entered by the user

https://gyazo.com/efec0d29233d5bf02cebc414dc3ed911

https://gyazo.com/7599d7ac213ae267b7051d1165de22d1

Image caption generation for vector tile maps

OpenStreetMap standard style

https://gyazo.com/8326800078a1420491ae266129e4be8a

Intentionally simplified style

https://gyazo.com/264fef8837e24a25095e980ba575ebd4

https://gyazo.com/a32b169ece3be59dc8bacbd69124e8b7